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DETAILED ACTION 



Response to Amendment 

1. The amendment filed 15 September 2005 has been entered in full. 

2. Based on applicant's amendments to claims, the objections on the respective claims have 
been withdrawn. 

3. Applicant's amendment to the drawing 4 has been fully entered, and is persuasive. Objection 
on drawing 4 has been withdrawn. 

4. Applicant's amendments to the claims have been fully considered, but are moot in view of 
the new ground(s) of rejection. The indicated allowability of claims 37 and 38 is withdrawn in view 
of the newly discovered reference(s) to Katto, U.S. Patent No. 5,917,936, and therefore the finality 
of the last office action mailed to the applicant is withdrawn. Rejections based on the newly cited 
reference(s) follow. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms die basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to 
which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made. 



6. Claims 1, 3-4, 6-13, 20-33 and 37-42 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Baxes, Book Publication, 1994, "Digital image processing: principles and 
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application", further in view of Katto, U.S. Patent No. 5,917,936, and further in view of Ishikawa et 
al., U.S. Patent No. 4,625,329. 

Regarding claim 1, Baxes teaches that the image segmentation process consists of three 
stages (i) image preprocessing, where the image is visually improved to make it as free of zero- 
information-carrying clutter as possible, (ii) initial object discrimination, where objects are grossly 
separated into groups with similar attributes, and (iii) object boundary cleanup which includes 
image morphological processing, where object boundaries are reduced to single-pixel widths and 
removes noise clutter and other artifacts from the image (page 124, para. 2, figure 5.1). Baxes further 
discloses that an image exists as a large array of pixels with associated brightness (characteristic) 
(page 123, para. 2, line 1). It is clear from this disclosure by Baxes that image to be segmented 
consists of a plurality of pixels and each pixel inherendy has a value (initial value) associated with it 
to measure the characteristics associated which conforms to the limitation "receiving an ambient 
image represented by a plurality of pixels and plurality of initial pixel values, wherein each 
said pixel has at least one said initial pixel value". 

Baxes teaches the second stage (initial object discrimination) of the image segmentation 
process, which uses binary contrast enhancement operation to isolate objects in the image, by 
highlighting similar objects with a common brightness (page 126, para. 3). Baxes further discloses 
binary contrast enhancement consists of comparing each pixel (initial) value to the threshold value 
to identify whether pixel belongs to darker (background) area or the brighter (foreground or object) 
area where darker and brighter areas are two pixel categories (page 330) which conforms to the 
limitation "identifying one or more pixels as belonging to one of a plurality of pixel 
categories on the basis of one or more initial pixel values associated with the pixels being 
identified". 
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Baxes further teaches binary contrast enhancement process consists the comparing of pixel 
brightness (initial) value to the brightness threshold value where if the pixel brightness is darker than 
the threshold, the resulting pixel brightness is set to 0 (first revised pixel value) and if the pixel 
brightness is brighter than the threshold, the resulting brightness is set to 255 (first revised pixel 
value) (page 330, para 1). Baxes further discloses determining (generating) an image threshold 
(page 77, para. 2, lines 3-5 reciting "the adaptive threshold value is usually determined by 

computing the average brightness in the area "). It is clear from this disclosure that each 

pixel in the brighter area will have a value of 255 and each pixel in the darker area will have a value 
of 0 which conforms to the limitation "establishing a first revised pixel value for one or more 
pixels, wherein each pixel in the same pixel category has the same first revised pixel value". 

Baxes teaches the third stage (object boundary cleanup) of the image segmentation 
process, which uses a binary morphological processes (operations) such as erosion and dilation 
(opening and closing), to operate upon binary images, such as those created using a binary contrast 
enhancement operation (pagel27, para. 2; page 128, para. 4 and 5). Baxes further teaches of 
comparing every pixel in the image with its 8 neighbors to produce a resulting output pixel value 
(second revised pixel value) (page 128, last para.). Baxes further teaches "In case where all nine input 
pixels are identical to their respective mask values, the resulting pixel value is set to a predefined 
logical value (either 0 or 1). Where one or more input pixels do not match their respective mask 
values, the resulting value is set to the opposite state (revised second pixel value) (page 129, para.l) 
which conforms to the limitation "setting a second revised pixel value for one or more of said 
pixels on the basis of one or more first revised pixel values that are associated with one or 
more pixels in the vicinity of the pixel being set". 
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Baxes as discussed above performs image segmentation using three stages where 2 nd stage 
(initial object discrimination) establishes the first revised value of the pixels and 3 rd stage (object 
boundary cleanup) establishes the second revised value of the pixels where Baxes establishes 
second revised value of the pixels using morphological heuristic. However, the amended claim 1 
recites the additional limitation "wherein setting the second revised pixel value includes at least one 
of a momentum-based heuristic and a gravity-based". As discussed before, Baxes establishes second 
revised value of the pixels using morphological heuristic in the third stage (object boundary 
cleanup), where object boundary cleanup which includes image morphological processing, where 
object boundaries are reduced to single-pixel widths and removes noise clutter and other 
artifacts from the image (page 124, para. 2, figure 5.1), but does not teach object boundary cleanup 
using at least one of a momentum-based heuristic and a gravity-based heutristic. Based on 
examiner's additional search, examiner finds the reference Katto. Katto discloses gravity-based 
heuristic used in image segmentation (col. 1, lines 9-40) by disclosing "This proposed clustering 
technique takes into account the five-dimensional space including the tristimulus values R, G and B 
as a feature amount space and a pixel position (x, y). Here, assume that a color information of an 

attentive pixel is (R, G, B), a position information for the attentive pixel is (x, y), and a 

positional gravity of the cluster at the index Furthermore, the distance d n is calculated for a 

plurality of clusters in the neighborhood of the attentive pixel, and it is concluded that the attentive 
pixel belongs to the cluster "n" having a minimum distance d n ", thus assigning the revised value to 
the pixel belonging to the cluster having a minimum distance. Therefore, it would have been 
obvious for one of ordinary skill in the art at the time of invention was made to use Katto teachings 
of using Gravity-based heuristic to establish the second revised pixel value in the invention of Baxes 
because gravity based heuristic would provide region based segmentation approach without losing 
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local information and is robust when presented with noisy input and is not sensitive to edge 
definition (See Katto, col. 1, lines 38-41). 

Also, Katto further discloses momentum-based heuristic (col. 1, lines 45-68 through col. 2, 
lines 1-15). 

The whole disclosure by Baxes in view of Katto, discussed above would conform to the 
limitation "deriving a segmented image of the occupant (object) from said first revised pixel 
value and said second revised pixel value". 

Baxes further discloses "the mission of image analysis operations (such as image 
segmentation) is to understand an image by quantifying its elements. The elements of interest are 
generally objects in an image, such as cells, bolts, aircraft, or characters on a page of text, which are 
occupant of the image. Image analysis operations play a major role in automated machine vision 
and image interpretation applications of digital image processing" (page 123, para. 1). It is clear 
form above disclosure that image processing operations as disclosed by Baxes can be used in any 
image processing field where image segmentation is required to identify the object (occupant) in the 
image and this is further supported by Ishikawa. Ishikawa discloses of a similar image segmentation 
method and system, which includes an image sensor for generating sensor measurements to identify 
the occupant in the vehicle (figures 8, 10A and 10B; col. 1, lines 56-60; col. 2, lines 28-32 and lines 
45-46; col.5, lines 15-50). 

Therefore, it would have been obvious for one of ordinary skill in the art at the time of the 
invention was made to use combined Baxes's and Katto's teachings in the machine vision and image 
interpretation applications related to image processing such as disclosed in Ishikawa's invention. 
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Claim 3 recites "an image segmentation method as in claim 1, wherein identifying one or 
more said pixels further includes comparing said plurality of initial pixel values to said image 
threshold". Claim 3 has been similarly analyzed and rejected as per the rejection. of claim 1. 

Claim 4 recites "an image segmentation method as in claim 1 , wherein generating an image 
threshold further comprises analyzing the distribution of initial pixel values relating to a pixel 
characteristic". Baxer discloses determining the threshold by computing the average brightness in 
the area (page 77, lines 3-4; page 153, para. 3) and further teaches that brightness and color features 
(pixel characteristics) of an object can be extracted by examining every pixel within the object's 
boundaries and discloses that this can be done using a histogram, where histogram shows the 
brightness (pixel characteristic) distribution found in the object (page 153, para. 2 and 3). 

Claim 6 recites "an image segmentation method as in claim 4, wherein each pixel has only 
one initial pixel value and only one pixel characteristic". Baxes as discussed in the rejection of claim 
4, uses a histogram where histogram provides the characteristic (brightness) associated with each 
pixel, It is clear from this that in histogram graph (page 63, figure 3.19b) each pixel will have one 
value for each characteristic (sharpness) associated with it. 

Claim 7 recites "an image segmentation method as in claim 6, wherein luminosity is said 
pixel characteristic". Luminosity is the synonym of Brightness as the characteristic of the pixel. 
Claim 7 has been similarly analyzed and rejected as per claims 6, 4 and 1. 
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Claim 8 recites "an image segmentation method as in claim 3, wherein each said pixel has a 
pixel location with respect to the ambient image, and said pixel location determines which of a 
plurality of image thresholds are compared to said initial pixel value for said pixel in said pixel 
location". As disclosed in the rejection of claim 1, Baxes teaches that an image exists as a large array 
of pixels with associated brightness (characteristic) (page 123, para. 2, line 1). It is clear from this 
disclosure by Baxes that image is made of plurality of pixels and each pixel has to have a any pixel 
location to represent the image and the position of the pixel can further be determined using 
histogram which shows the distribution of image pixels. Baxes further discloses in the rejection of 
claim 1, that for image segmentation the initial pixel value of the pixel is compared to the image 
threshold generated. Baxes further teaches adaptive thresholding in which the appropriate 
threshold value may need to change throughout the image being processed depending on 
the qualities of the image in the area being processed, for example, "in areas where the 
image is darker, the threshold value decrease and in areas where the image is lighter 
(brighter), the threshold value increases" (page 76, para. 1). It is clear from the above 
disclosure by Baxes, adaptive thresholding can be used to determine a plurality of image thresholds 
depending on the pixel location, to which the each said pixel can be compared to, which conforms 
to the limitation "said pixel location determines which of a plurality of image thresholds are 
compared to said initial pixel value for said pixel in said pixel location". 

Claim 9 recites "an image segmentation method as in claim 8, wherein a higher image 
threshold is applied in pixel locations where there is brighter lighting". Claim 9 has been similarly 
analyzed and rejected as per claim 8. 
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Claim 10 recites "an image segmentation method as in claim 1, wherein there are only two 
pixel categories". As disclosed by Baxes in the rejection of claim 1, The pixel is either identified as 
belonging to darker area or brighter area where darker and brighter areas are two pixel categories. 
Claim 10 has been similarly analyzed and rejected as per claim 1. 

. Claim 11 recites "an image segmentation method as claim 1, wherein setting die second 
revised pixel value includes a morphological heuristic". As disclosed by Baxes in rejection of claim 1, 
the third stage (object boundary cleanup) of the image segmentation process, which uses a binary 
morphological processes (heuristic) such as erosion and dilation, to operate upon binary images, 
such as those created using a binary contrast enhancement operation (pagel27, para. 2; page 128, 
para. 4 and 5) and Baxes further teaches of comparing every pixel in the image with its 8 neighbors 
to produce a resulting output pixel value (second revised pixel value) (page 128, last para.). Claim 11 
has been similarly analyzed and rejected as per claim 1. 

Claim 12 recites "an image segmentation method as in claim 11, wherein the morphological 
heuristic is a morphological erosion". Claim 12 has been similarly analyzed and rejected as per claim 
1 1 and claim 1 . 

Claim 13 recites "an image segmentation method as in claim 12, wherein the morphological 
heuristic is a morphological dilation". Claim 13 has been similarly analyzed and rejected as per claims 
12, 11 and 1. 
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Claim 20 recites an image segmentation system, which comprises of an image thresholding 
subsystem and a gap processing subsystem. As discussed in the rejection of claim 1, Baxes discloses 
image segmentation, which comprises of three stages where second stage provides the image 
thresholding and third stage uses dilation and erosion (closing and opening) for a gap processing 
(page 136, para. 5, lines 2-3). To perform these image segmentation operations, a system is obviously 
required. Ishikawa presents a image processing system for image segmentation as discussed in claim 
1. 

Claim 21 has been similarly analyzed and rejected as per claims 20, 1, 4, 6 and 7. 

Claim 22 has been similarly analyzed and rejected as per claims 21, 20, 1, 4, 6, and 7. 

Claim 23 has been similarly analyzed and rejected as per claims 22 -> 20 and 5. 

Claim 24 has been similarly analyzed and rejected as per claims 23 -> 20 and 5. 

Claim 25 has been similarly analyzed and rejected as per claims 24 -> 20 and 8. 

Claim 26 recites two subsets of pixels similarly like two categories (darker area and brighter 
area which contain dark and white pixels) as in rejection of claim 1. Claim 26 has been similarly 
analyzed and rejected as per claims 24 ~> 20 and claim 1. 



Claim 27 has been similarly analyzed and rejected as per claims 26 -> 20 and claim 1. 
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Claim 28 recites "an image segmentation system as in claim 27 wherein at least 
approximately half of said plurality of said pixels are set to said first binary value". Setting the 
plurality of said pixels to said first binary value depends on the size of the object to be segmented 
and the on the area of the background, if the area of the object is half the image, then it is clear that 
almost half the pixels in the image will be set to binary value to be labeled to the object and rest to 
the background. 

Claim 29-31 have been similarly analyzed and rejected as claims 20, 1 and 11-13. 

Claim 32 recites "an image segmentation as in claim 29, said morphological heuristic 
comprising a vertically-based morphological heuristic'\ Baxes further discloses that morphological 
erosion and dilation operations can b performed in vertical and horizontal directions (page 136, 
para. 1, line 3). 

Claim 33 recites "an image segmentation as in claim 29, said morphological heuristic 
comprising a horizontally-based morphological heuristic". Claim 33 has been similarly analyzed and 
rejected as per claim 32. 

Claims 37 and 38 have been similarly analyzed and rejected as per die claim 1. Emphasis is 
made on the disclosure provided by Katto. 
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Claims 39-42 recites the use of different thresholds in different areas of the image thus 
providing an adaptive thresholding and it is well known to be used in the art of image processing 
and further emphasis has been added in the arguments as disclosed in the rejection of claim 8 which 
shows "Baxes teaches adaptive thresholding in which the appropriate threshold value may need to 
change throughout the image being processed depending on the qualities of the image in the area 
being processed, for example, "in areas where the image is darker, the threshold value decrease and 
in areas where the image is lighter (brighter), the threshold value increases" (page 76, para. 1). 
Therefore claims 39-42 has been similarly analyzed and rejected as per claim 8. 

7. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Baxes, Book 
Publication, 1994, "Digital image processing: principles and application", further in view of Katto, 
U.S. Patent No. 5,917,936, and further in view of Ishikawa et al, U.S. Patent No. 4,625,329 and 
further in view of Google article, Jan. 29, 2001, "Point operations", and further in view of Hentea, 
IEEE Publication, 1993, "Algorithm for automatic threshold determination for image 
segmentation". 

Claim 5 recites "an image segmentation method as in claim wherein analyzing the 
distribution of initial pixel values further includes: recording aggregate initial pixel values into a 
histogram; translating the histogram into a cumulative distribution function". As disclosed in the 
rejection of claims 1 and 4, Baxes discloses the use of histogram to generate the image threshold by 
recording the initial pixel values of the image in the histogram. Baxes does not teach of translating 
the histogram into a cumulative distribution function and calculating an image threshold based on 
the predetermined percentage of initial pixel values falling below the image threshold and neither 
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does Ishikawa teaches the same. It is well known in the art of image processing that thresholding in 
its various versions (local, global, and dynamic), is the most commonly used method for image 
segmentation and global thresholding produces satisfactory results if the input images are corrected 
for uneven llluminant (shading correction). Translating the histogram into a cumulative distribution 
function is nothing but histogram equalization and histogram equalization is well known in the art of 
image processing and is further supported by Google article "Point operations". This article on page 
2 of 4, discloses histogram equalization by obtaining the cumulative probability distribution of r 
(where r is an input value). Therefore, it would have been obvious for one of ordinary skill in the art 
at the time of invention was made, to use the teachings of "Point operations" of obtaining the 
cumulative probability distribution of the histogram in the combined invention of Baxes, Katto and 
Ishikawa. One would have been motivated to use the teachings of "Point operations" of obtaining 
the cumulative probability distribution of the histogram in the combined invention of Baxes, Katto 
and Ishikawa because both the all the reference are directed to image processing field and "Point 
operations" performs histogram equalization by obtaining the cumulative probability distribution of 
the histogram to distribute the gray levels of the image evenly over the maximum possible 
range (page 2 of 4). 

Claim 5 further recites the limitation "calculating an image threshold based on a 
predetermined percentage of initial pixel values falling below the image threshold". Baxes does teach 
of determining image threshold based on the calculation performed on the initial pixel values but 
calculating an image threshold based on the predetermined percentage of initial pixel values falling 
below the image threshold and neither does Ishikawa and "Point operations" teaches the same. 
However, Hentea discloses an algorithm for automatic threshold selection for the detection of 
objects that are darker than the background (page 535, right col, para. 2). Hentea further provides 
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the teachings where a cumulative histogram is calculated from the pixel values and then the image 
threshold is automatically selected as the gray level for which a given percentage of the blocks (or 
plurality of pixel values) have minimum values lower than it (image threshold) (page 535, right col., 
para. 4). 

Therefore, it would have been obvious for one of ordinary skill in the art at the time of the 
invention was made to use the teachings of Hentea in the combined invention of Baxes, Katto, 
Ishikawa and "Point operations". One would have been motivated to use the teachings of Hentea in 
the combined invention of Baxes, Katto, Ishikawa and "Point operations" because Hentea belongs 
to the same art of endeavor (image segmentation) as are Baxes, Katto and Ishikawa and Hentea 
further provides the teachings for the automatic selection of the threshold for global thresholding 
which is very robust and performs well for images belonging to a large class and is also very efficient 
from a computation point of view, and is very flexible (page 538, right col, para. 1). 

Allowable Subject Matter 

Reasons for Allowance: 

8. Claims 18 and 19 are allowed. 

The following is an examiner's statement of reasons of allowance: 

. Both the instant invention and the closest prior art Baxes, Book Publication, 1994, 
"Digital image processing: principles and application" are directed to a method of image 
segmentation to identify objects (occupants) in the images. The instant invention further recites the 
limitations "determining a third revised pixel value from said second revised pixel value or said first 
revised pixel value with a momentum-based heuristic; identifying regions of pixels based on the first 
revised pixel value, the second revised pixel value, and the third revised pixel value" in claim 18 
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whereas Baxes and other prior art of record does not teach use of momentum-based operations or 
heuristic to identify regions of pixels based on the first revised pixel value, the second revised pixel 
value, and the third revised pixel value and so on. Therefore claim 18 is allowed. Claim 19 is 
dependent on claim 18 and therefore is allowed. 

9. Claims 15-16 are objected to as being dependent upon a rejected base claim 1, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and any 
intervening claims. 

1 0. Claims 34-36 are objected to as being dependent upon a rejected base claim 20, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and any 
intervening claims. 

Conclusion 

11. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Examiner asserts these references clearly teach image segmentation using gravity-based 
heuristics and momentum-based heuristics. 

• Oddo et al, IEEE Publication, 2000, "Automated road extraction via the 
hybridization of self-organization and model based techniques". More emphasis is 
added to page 33, left column, 2 nd -4 lh para which provides gravity based heuristics for 
image segmentation to remove clutter and obscuration. 
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• Collet et al, IEEE Publication, 1994, "Active contour models for infrared cloudy 
shapes segmentation". More emphasis is added to Abstract and conclusion. 

• Kikuchi et al, IEEE Publication, 1999, "Recognition process using feature data 
fusion for imaging system". 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Manav Seth whose telephone number is (571) 272-7456. The examiner can 
normally be reached on Monday to Friday from 8:30 am to 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiners supervisor, 
Bhavesh Mehta, can* be reached on (571) 272-7453. The fax phone number for the organization 
where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR system, 
see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, 
contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Manav Seth 
Art Unit 2625 
September 30, 2005 



